A Cross-Jobs-Cross-Phases Map-Reduce Scheduling Algorithm in Heterogeneous Cloud

نویسندگان

  • Lei Chen
  • Jing Zhang
  • Li-Jun Cai
  • Zi-Yun Deng
  • Tao Meng
چکیده

To fast process the large-scale data, map-reduce cloud is viewed as a very reasonable and effective platform. According to the new scheduling challenges in map-reduce cloud, a cross-jobs-cross-phases (CJCP) map-reduce scheduling algorithm is proposed in this paper. CJCP mainly consists of four optimal schemes, and respectively deals with four resource waste scenes of the job scheduling process. In the first scene, based on the job training method, an optimal scheme is designed to shield the interference of heterogeneous resources on job scheduling. In the second scene, we give two definition and develop another optimal scheme to dynamically adjust task number of multiple virtual machines on the same physical host. Through task adjustment, the high-capacity virtual machines deal more tasks than low-capacity ones. In addition, we build the overlapping execution model to overlap map, shuffle and reduce phases. In the third scene, considering the difference of map tasks and push tasks on resource usage needs, an overlapping scheme is formed to optimal the execution of push and map phase. In the last scene, a cross-jobs optimal scheme is proposed, which overlap execution of current job and next job. To avoid the bandwidth confliction between two jobs, a monitor method is used for reasonably resources allocation. Extensive experiments show that our algorithm consumes less job execution time and performs better than other three algorithms.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Cross-layer Packet-dependant OFDM Scheduling Based on Proportional Fairness

This paper assumes each user has more than one queue, derives a new packet-dependant proportional fairness power allocation pattern based on the sum of weight capacity and the packet’s priority in users’ queues, and proposes 4 new cross-layer packet-dependant OFDM scheduling schemes based on proportional fairness for heterogeneous classes of traffic. Scenario 1, scenario 2 and scenario 3 lead r...

متن کامل

Task Scheduling Algorithm Using Covariance Matrix Adaptation Evolution Strategy (CMA-ES) in Cloud Computing

The cloud computing is considered as a computational model which provides the uses requests with resources upon any demand and needs.The need for planning the scheduling of the user's jobs has emerged as an important challenge in the field of cloud computing. It is mainly due to several reasons, including ever-increasing advancements of information technology and an increase of applications and...

متن کامل

A new Shuffled Genetic-based Task Scheduling Algorithm in Heterogeneous Distributed Systems

Distributed systems such as Grid- and Cloud Computing provision web services to their users in all of the world. One of the most important concerns which service providers encounter is to handle total cost of ownership (TCO). The large part of TCO is related to power consumption due to inefficient resource management. Task scheduling module as a key component can has drastic impact on both user...

متن کامل

Integrated modeling and solving the resource allocation problem and task scheduling in the cloud computing environment

Cloud computing is considered to be a new service provider technology for users and businesses. However, the cloud environment is facing a number of challenges. Resource allocation in a way that is optimum for users and cloud providers is difficult because of lack of data sharing between them. On the other hand, job scheduling is a basic issue and at the same time a big challenge in reaching hi...

متن کامل

Trucks Scheduling to Reduce Delays in Multiple Cross-Docks Using Imperialist Competitive Algorithm

Cross docking is a logistics strategy that is now used by a large number of firms in the various industries. Cross docking effectively a significant reduction in transport costs, without increasing inventory and at the same time maintain the level of service to customers. It also can cross dock order to reduce cycle time, improve flexibility and responsiveness lead distribution network. In this...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:

دوره   شماره 

صفحات  -

تاریخ انتشار 2017